The present invention relates generally to stereo imaging, and more particularly, to techniques for correcting an epipolar axis for skew and offset in order to approximate a horizontal axis in disparity calculations.
The art of stereo imaging involves the generation and analysis of at least two images of an object from different points of view. Typically, the two images correspond to left and right images generated by cameras positioned along, and orthogonal to, at least two viewpoints on a horizontal axis. Collectively, the left and right images are called a stereo image pair.
The left and right images of a stereo image pair may be analyzed to yield disparity data. Disparity is the amount of translation along a conceptual epipolar axis that all objects move along in response to changing the image creation viewpoint. If the cameras are perfectly aligned, both vertically and rotationally, the epipolar axis is identical to the horizontal axis.
For example, objects in the left view of a stereo image pair appear to be further to the right along the epipolar axis than the same objects in the right image. Moreover, objects closer to the cameras will appear to be horizontally shifted along the epipolar axis to a greater degree than objects farther from the cameras. In extreme cases, an object in the distant horizon, such as the sun, may not appear to shift at all, whereas objects immediately in front one camera may appear centered in one image but not even appear in the other image.
Disparity data is useful in numerous applications ranging from distance calculation to edge detection. For example, using well known techniques, the degree of disparity between the location of an object in two images may be used to calculate the distance of the object from the cameras.
Moreover, disparity data may be used to locate the edges of an object represented in a stereo image pair. Because the background of an object is farther away from the cameras than the object itself, the background exhibits a lesser degree of disparity than does the object. Consequently, the difference in disparity distinguishes the edges of the object from the background.
In general, disparity is calculated by matching blocks of picture elements (pixels) along the same horizontal rows of the left and right images. This technique relies, sometimes correctly, on the assumption that the epipolar axis is approximately equal to the horizontal axis. Once the matching blocks are identified, the amount of horizontal translation, and hence, the degree of disparity, may be determined.
Unfortunately, the horizontal axis is often a poor approximation of the epipolar axis. For example, the left camera may be slightly rotated with respect to the right camera, resulting in a left image that is skewed with respect to the right image. Likewise, the left camera may be slightly higher or lower than the right camera, resulting in a left image that is vertically offset from the right image.
Skew and offset cause the epipolar axis to deviate from the horizontal axis. Thus, if blocks of pixels are compared along the same horizontal rows in the left and right images, no matching blocks are found. Without a compensating calculation, the degree of disparity cannot be determined.
Conventional methods for correcting an epipolar axis for skew and offset require vast amounts of calculation. For example, using standard techniques, the precise amount of skew and offset for each image may be determined, and appropriate transformations, including image rotations and translations, may be performed. However, these transformations often require days of CPU time for high-resolution images and make real-time disparity calculations impracticable.
Accordingly, a need exists for an improved system and method for quickly compensating for skew and offset of an epipolar axis to allow for rapid determination of disparity values.
The present invention solves the foregoing problems by providing an improved system and method for correcting an epipolar axis for skew and offset in order to approximate a horizontal axis in disparity calculations.
In one aspect of the invention, cameras (10, 12) produce first and second images (14, 16) of an object (18) from different viewpoints. An image partitioning module (54) partitions the images (14, 16) into a plurality of vertically striped regions (70), such that each region (70) of the first image (14) corresponds to a region (70) of the second image (16). A region alignment module (55) vertically shifts a region (70) of the first image (14) in a direction calculated to vertically align a portion (30) of the region (70) with a substantially matching portion (32) of the corresponding region (70).
In another aspect of the invention, the region alignment module (55) includes a block matching module (56), an offset calculation module (58), and a region shifting module (60). Where a first region (70) of the first image (14) corresponds to a second region (70) of the second image (16), the block matching module (56) identifies a portion (32) of the second region (70) that substantially matches a portion (30) of the first region (70). The offset calculation module (58) calculates a vertical offset between the substantially matching portions (30,32). The region shifting module (60) vertically shifts the first region (70) according to the calculated vertical offset in order to vertically align the substantially matching portions (30, 32) of the first and second regions (70).
In yet another aspect of the invention, a computer-implemented method includes the steps of partitioning the first and second images (14,16) into a plurality of vertically striped regions (70) and vertically shifting a region (70) of the first image (14) in a direction calculated to vertically align a portion (30) of the region (70) with a substantially matching portion (32) of the corresponding region (70).
In still another aspect of the invention, the vertically shifting step includes the steps of identifying a portion (32) of the second region (70) that substantially matches a portion (30) of the first region (70); calculating a vertical offset between the substantially matching portions (30, 32); and vertically shifting the first region (70) according to the calculated vertical offset in order to vertically align the substantially matching portions (30, 32) of the first and second regions (70).
In another aspect of the invention, an article of manufacture comprises a program storage medium readable by a processor (42) and embodying one or more instructions executable by the processor (42) to perform the above-described method.